Deep Reinforcement Learning Sensor Scheduling for Effective Monitoring of Dynamical Systems.

Advances in technology have enabled the use of sensors with varied modalities to monitor different parts of systems, each providing diverse levels of information about the underlying system. However, resource limitations and computational power restrict the number of sensors/data that can be processed in real-time in most complex systems. These challenges necessitate the need for selecting/scheduling a subset of sensors to obtain measurements that guarantee the best monitoring objectives. This paper focuses on sensor scheduling for systems modeled by hidden Markov models. Despite the development of several sensor selection and scheduling methods, existing methods tend to be greedy and do not take into account the long-term impact of selected sensors on monitoring objectives. This paper formulates optimal sensor scheduling as a reinforcement learning problem defined over the posterior distribution of system states. Further, the paper derives a deep reinforcement learning policy for offline learning of the sensor scheduling policy, which can then be executed in real-time as new information unfolds. The proposed method applies to any monitoring objective that can be expressed in terms of the posterior distribution of the states (e.g., state estimation, information gain, etc.). The performance of the proposed method in terms of accuracy and robustness is investigated for monitoring the security of networked systems and the health monitoring of gene regulatory networks.

Modeling Defensive Response of Cells to Therapies: Equilibrium Interventions for Regulatory Networks.

A major objective in genomics is to design interventions that can shift undesirable behaviors of such systems (i.e., those associated with cancers) into desirable ones. Several intervention policies have been developed in recent years, including dynamic and structural interventions. These techniques aim at making targeted changes to cell dynamics upon intervention, without considering the cell's defensive mechanisms to interventions. This simplified assumption often leads to early and short-term success of interventions, followed by partial or full recurrence of diseases. This is due to the fact that cells often have dynamic and intelligent responses to interventions through internal stimuli. This paper models gene regulatory networks (GRNs) using the Boolean network with perturbation. The dynamic and adaptive battle between intervention and the cell is modeled as a two-player zero-sum game, where intervention and the cell fight against each other with fully opposite objectives. An optimal intervention policy is obtained as a Nash equilibrium solution, through which the intervention is stochastic, ensuring the optimal solution to all potential cell responses. We analytically analyze the superiority of the proposed intervention policy against existing intervention techniques. Comprehensive numerical experiments using the p53-MDM2 negative feedback loop regulatory network and melanoma network demonstrate the high performance of the proposed method.

Modeling the structural relationships between trauma exposure with substance use tendency, depression symptoms, and suicidal thoughts in individuals with earthquake trauma experience: the mediatory role of peritraumatic dissociation and experiential avoidance.

BACKGROUND: Despite the fact that studies indicate that earthquake trauma is associated with numerous psychological consequences, the mediating mechanisms leading to these outcomes have not been well-studied. Therefore, this study investigates the relationship between trauma exposure with substance use tendency, depression, and suicidal thoughts, with the mediating role of peritraumatic dissociation and experiential avoidance. METHODS: The descriptive-correlational approach was employed in this study. The participants were people who had experienced the Kermanshah earthquake in 2017. A total of 324 people were selected by convenient sampling method. The Traumatic Exposure Severity Scale, the Peritraumatic Dissociative Experiences Questionnaire, the Acceptance and Action Questionnaire, the Iranian Addiction Potential Scale, Beck's Depression Inventory [BDI-II], and Beck's Suicidal Thoughts Scale were used to collect data. The gathered data was analyzed| using structural equation modeling in |SPSS Ver. 24 and LISREL Ver. 24. RESULTS: The study findings indicated that the intensity of the trauma exposure is directly and significantly associated with depression symptoms, peritraumatic dissociation, and experiential avoidance. The severity of exposure to trauma had a significant indirect effect on the tendency to use substances through experiential avoidance. This is while the severity of the trauma experience did not directly correlate with substance use and suicidal thoughts. In addition, peritraumatic dissociation did not act as a mediator in the relationship between the severity of trauma exposure with substance use, depression, and suicidal thoughts. CONCLUSIONS: The severity of exposure to the earthquake was associated with symptoms of depression and these findings indicate the importance of experiential avoidance in predicting the tendency to use drugs. Hence, it is essential to design and implement psychological interventions that target experiential avoidance to prevent drug use tendencies and to establish policies that lower depression symptoms following natural disasters.

Navigating the Landscape of Translational Geroscience in Canada: A Comprehensive Evaluation of Current Progress and Future Directions.

The inaugural Canadian Conferences on Translational Geroscience were held as two complementary sessions in October and November 2023. The conferences explored the profound interplay between the biology of aging, social determinants of health, the potential societal impact of geroscience and the maintenance of health in aging individuals. Although topics such as cellular senescence, molecular and genetic determinants of aging and prevention of chronic disease were addressed, the conferences went on to emphasize practical applications for enhancing older people's quality of life. This manuscript summarizes the proceeding and underscores the synergy between clinical and fundamental studies. Future directions highlight national and global collaborations and the crucial integration of early-career investigators. This work charts a course for a national framework for continued innovation and advancement in translational geroscience in Canada.

Classification of Fracture Risk in Fallers Using Dual-Energy X-Ray Absorptiometry (DXA) Images and Deep Learning-Based Feature Extraction.

Dual-energy X-ray absorptiometry (DXA) scans are one of the most frequently used imaging techniques for calculating bone mineral density, yet calculating fracture risk using DXA image features is rarely performed. The objective of this study was to combine deep neural networks, together with DXA images and patient clinical information, to evaluate fracture risk in a cohort of adults with at least one known fall and age-matched healthy controls. DXA images of the entire body as, well as isolated images of the hip, forearm, and spine (1488 total), were obtained from 478 fallers and 48 non-faller controls. A modeling pipeline was developed for fracture risk prediction using the DXA images and clinical data. First, self-supervised pretraining of feature extractors was performed using a small vision transformer (ViT-S) and a convolutional neural network model (VGG-16 and Resnet-50). After pretraining, the feature extractors were then paired with a multilayer perceptron model, which was used for fracture risk classification. Classification was achieved with an average area under the receiver-operating characteristic curve (AUROC) score of 74.3%. This study demonstrates ViT-S as a promising neural network technique for fracture risk classification using DXA scans. The findings have future application as a fracture risk screening tool for older adults at risk of falls. © 2023 The Authors. JBMR Plus published by Wiley Periodicals LLC on behalf of American Society for Bone and Mineral Research.

Learning to Fight Against Cell Stimuli: A Game Theoretic Perspective.

Current genomics interventions have limitations in accounting for cell stimuli and the dynamic response to intervention. Although genomic sequencing and analysis have led to significant advances in personalized medicine, the complexity of cellular interactions and the dynamic nature of the cellular response to stimuli pose significant challenges. These limitations can lead to chronic disease recurrence and inefficient genomic interventions. Therefore, it is necessary to capture the full range of cellular responses to develop effective interventions. This paper presents a game-theoretic model of the fight between the cell and intervention, demonstrating analytically and numerically why current interventions become ineffective over time. The performance is analyzed using melanoma regulatory networks, and the role of artificial intelligence in deriving effective solutions is described.

Structure-Based Inverse Reinforcement Learning for Quantification of Biological Knowledge.

Gene regulatory networks (GRNs) play crucial roles in various cellular processes, including stress response, DNA repair, and the mechanisms involved in complex diseases such as cancer. Biologists are involved in most biological analyses. Thus, quantifying their policies reflected in available biological data can significantly help us to better understand these complex systems. The primary challenges preventing the utilization of existing machine learning, particularly inverse reinforcement learning techniques, to quantify biologists' knowledge are the limitations and huge amount of uncertainty in biological data. This paper leverages the network-like structure of GRNs to define expert reward functions that contain exponentially fewer parameters than regular reward models. Numerical experiments using mammalian cell cycle and synthetic gene-expression data demonstrate the superior performance of the proposed method in quantifying biologists' policies.

Reinforcement Learning Data-Acquiring for Causal Inference of Regulatory Networks.

Gene regulatory networks (GRNs) consist of multiple interacting genes whose activities govern various cellular processes. The limitations in genomics data and the complexity of the interactions between components often pose huge uncertainties in the models of these biological systems. Meanwhile, inferring/estimating the interactions between components of the GRNs using data acquired from the normal condition of these biological systems is a challenging or, in some cases, an impossible task. Perturbation is a well-known genomics approach that aims to excite targeted components to gather useful data from these systems. This paper models GRNs using the Boolean network with perturbation, where the network uncertainty appears in terms of unknown interactions between genes. Unlike the existing heuristics and greedy data-acquiring methods, this paper provides an optimal Bayesian formulation of the data-acquiring process in the reinforcement learning context, where the actions are perturbations, and the reward measures step-wise improvement in the inference accuracy. We develop a semi-gradient reinforcement learning method with function approximation for learning near-optimal data-acquiring policy. The obtained policy yields near-exact Bayesian optimality with respect to the entire uncertainty in the regulatory network model, and allows learning the policy offline through planning. We demonstrate the performance of the proposed framework using the well-known p53-Mdm2 negative feedback loop gene regulatory network.

Optimal Recursive Expert-Enabled Inference in Regulatory Networks.

Accurate inference of biological systems, such as gene regulatory networks and microbial communities, is a key to a deep understanding of their underlying mechanisms. Despite several advances in the inference of regulatory networks in recent years, the existing techniques cannot incorporate expert knowledge into the inference process. Expert knowledge contains valuable biological information and is often reflected in available biological data, such as interventions made by biologists for treating diseases. Given the complexity of regulatory networks and the limitation of biological data, ignoring expert knowledge can lead to inaccuracy in the inference process. This paper models the regulatory networks using Boolean network with perturbation. We develop an expert-enabled inference method for inferring the unknown parameters of the network model using expert-acquired data. Given the availability of information about data-acquiring objectives and expert confidence, the proposed method optimally quantifies the expert knowledge along with the temporal changes in the data for the inference process. The numerical experiments investigate the performance of the proposed method using the well-known p53-MDM2 gene regulatory network.

Inference of regulatory networks through temporally sparse data.

A major goal in genomics is to properly capture the complex dynamical behaviors of gene regulatory networks (GRNs). This includes inferring the complex interactions between genes, which can be used for a wide range of genomics analyses, including diagnosis or prognosis of diseases and finding effective treatments for chronic diseases such as cancer. Boolean networks have emerged as a successful class of models for capturing the behavior of GRNs. In most practical settings, inference of GRNs should be achieved through limited and temporally sparse genomics data. A large number of genes in GRNs leads to a large possible topology candidate space, which often cannot be exhaustively searched due to the limitation in computational resources. This paper develops a scalable and efficient topology inference for GRNs using Bayesian optimization and kernel-based methods. Rather than an exhaustive search over possible topologies, the proposed method constructs a Gaussian Process (GP) with a topology-inspired kernel function to account for correlation in the likelihood function. Then, using the posterior distribution of the GP model, the Bayesian optimization efficiently searches for the topology with the highest likelihood value by optimally balancing between exploration and exploitation. The performance of the proposed method is demonstrated through comprehensive numerical experiments using a well-known mammalian cell-cycle network.

Intermittent fasting and continuous energy restriction result in similar changes in body composition and muscle strength when combined with a 12 week resistance training program.

PURPOSE: The objective of this study was to compare the effects of 12 weeks of resistance training combined with either 5:2 intermittent fasting or continuous energy restriction on body composition, muscle size and quality, and upper and lower body strength. METHODS: Untrained individuals undertook 12 weeks of resistance training plus either continuous energy restriction [20% daily energy restriction (CERT)] or 5:2 intermittent fasting [~ 70% energy restriction 2 days/week, euenergetic consumption 5 days/week (IFT)], with both groups prescribed a mean of ≥ 1.4 g of protein per kilogram of body weight per day. Participants completed 2 supervised resistance and 1 unsupervised aerobic/resistance training combination session per week. Changes in lean body mass (LBM), thigh muscle size and quality, strength and dietary intake were assessed. RESULTS: Thirty-four participants completed the study (CERT = 17, IFT = 17). LBM was significantly increased (+ 3.7%, p < 0.001) and body weight (- 4.6%, p < 0.001) and fat (- 24.1%, p < 0.001) were significantly reduced with no significant difference between groups, though results differed by sex. Both groups showed improvements in thigh muscle size and quality, and reduced intramuscular and subcutaneous fat assessed by ultrasonography and peripheral quantitative computed tomography (pQCT), respectively. The CERT group demonstrated a significant increase in muscle surface area assessed by pQCT compared to the IFT group. Similar gains in upper and lower body strength and muscular endurance were observed between groups. CONCLUSION: When combined with resistance training and moderate protein intake, continuous energy restriction and 5:2 intermittent fasting resulted in similar improvements in body composition, muscle quality, and strength. ACTRN: ACTRN12620000920998, September 2020, retrospectively registered.

Two-Stage Bayesian Optimization for Scalable Inference in State-Space Models.

State-space models (SSMs) are a rich class of dynamical models with a wide range of applications in economics, healthcare, computational biology, robotics, and more. Proper analysis, control, learning, and decision-making in dynamical systems modeled by SSMs depend on the accuracy of the inferred/learned model. Most of the existing inference techniques for SSMs are capable of dealing with very small systems, unable to be applied to most of the large-scale practical problems. Toward this, this article introduces a two-stage Bayesian optimization (BO) framework for scalable and efficient inference in SSMs. The proposed framework maps the original large parameter space to a reduced space, containing a small linear combination of the original space. This reduced space, which captures the most variability in the inference function (e.g., log likelihood or log a posteriori), is obtained by eigenvalue decomposition of the covariance of gradients of the inference function approximated by a particle filtering scheme. Then, an exponential reduction in the search space of parameters during the inference process is achieved through the proposed two-stage BO policy, where the solution of the first-stage BO policy in the reduced space specifies the search space of the second-stage BO in the original space. The proposed framework's accuracy and speed are demonstrated through several experiments, including real metagenomics data from a gut microbial community.

Graph-Based Bayesian Optimization for Large-Scale Objective-Based Experimental Design.

Design is an inseparable part of most scientific and engineering tasks, including real and simulation-based experimental design processes and parameter/hyperparameter tuning/optimization. Several model-based experimental design techniques have been developed for design in domains with partial available knowledge about the underlying process. This article focuses on a powerful class of model-based experimental design called the mean objective cost of uncertainty (MOCU). The MOCU-based techniques are objective-based, meaning that they take the main objective of the process into account during the experimental design process. However, the lack of scalability of MOCU-based techniques prevents their application to most practical problems, including large discrete or combinatorial spaces. To achieve a scalable objective-based experimental design, this article proposes a graph-based MOCU-based Bayesian optimization framework. The correlations among samples in the large design space are accounted for using a graph-based Gaussian process, and an efficient closed-form sequential selection is achieved through the well-known expected improvement policy. The proposed framework's performance is assessed through the structural intervention in gene regulatory networks, aiming to make the network away from the states associated with cancer.

Scalable Inverse Reinforcement Learning Through Multifidelity Bayesian Optimization.

Data in many practical problems are acquired according to decisions or actions made by users or experts to achieve specific goals. For instance, policies in the mind of biologists during the intervention process in genomics and metagenomics are often reflected in available data in these domains, or data in cyber-physical systems are often acquired according to actions/decisions made by experts/engineers for purposes, such as control or stabilization. Quantification of experts' policies through available data, which is also known as reward function learning, has been discussed extensively in the literature in the context of inverse reinforcement learning (IRL). However, most of the available techniques come short to deal with practical problems due to the following main reasons: 1) lack of scalability: arising from incapability or poor performance of existing techniques in dealing with large systems and 2) lack of reliability: coming from the incapability of the existing techniques to properly learn the optimal reward function during the learning process. Toward this, in this brief, we propose a multifidelity Bayesian optimization (MFBO) framework that significantly scales the learning process of a wide range of existing IRL techniques. The proposed framework enables the incorporation of multiple approximators and efficiently takes their uncertainty and computational costs into account to balance exploration and exploitation during the learning process. The proposed framework's high performance is demonstrated through genomics, metagenomics, and sets of random simulated problems.

Validation of a Semiautomatic Image Analysis Software for the Quantification of Musculoskeletal Tissues.

Accurate quantification of bone, muscle, and their components is still an unmet need in the musculoskeletal field. Current methods to quantify tissue volumes in 3D images are expensive, labor-intensive, and time-consuming; thus, a reliable, valid, and quick application is highly needed. Tissue Compass is a standalone software for semiautomatic segmentation and automatic quantification of musculoskeletal organs. To validate the software, cross-sectional micro-CT scans images of rat femur (n = 19), and CT images of hip and abdomen (n = 100) from the Osteoporotic Fractures in Men (MrOS) Study were used to quantify bone, hematopoietic marrow (HBM), and marrow adipose tissue (MAT) using commercial manual software as a comparator. Also, abdominal CT scans (n = 100) were used to quantify psoas muscle volumes and intermuscular adipose tissue (IMAT) using the same software. We calculated Pearson's correlation coefficients, individual intra-class correlation coefficients (ICC), and Bland-Altman limits of agreement together with Bland-Altman plots to show the inter- and intra-observer agreement between Tissue Compass and commercially available software. In the animal study, the agreement between Tissue Compass and commercial software was r > 0.93 and ICC > 0.93 for rat femur measurements. Bland-Altman limits of agreement was - 720.89 (- 1.5e+04, 13,074.00) for MAT, 4421.11 (- 1.8e+04, 27,149.73) for HBM and - 6073.32 (- 2.9e+04, 16,388.37) for bone. The inter-observer agreement for QCT human study between two observers was r > 0.99 and ICC > 0.99. Bland-Altman limits of agreement was 0.01 (- 0.07, 0.10) for MAT in hip, 0.02 (- 0.08, 0.12) for HBM in hip, 0.05 (- 0.15, 0.25) for bone in hip, 0.02 (- 0.18, 0.22) for MAT in L1, 0.00 (- 0.16, 0.16) for HBM in L1, and 0.02 (- 0.23, 0.27) for bone in L1. The intra-observer agreement for QCT human study between the two applications was r > 0.997 and ICC > 0.99. Bland-Altman limits of agreement was 0.03 (- 0.13, 0.20) for MAT in hip, 0.05 (- 0.08, 0.18) for HBM in hip, 0.05 (- 0.24, 0.34) for bone in hip, - 0.02 (- 0.34, 0.31) for MAT in L1, - 0.14 (- 0.44, 0.17) for HBM in L1, - 0.29 (- 0.62, 0.05) for bone in L1, 0.03 (- 0.08, 0.15) for IMAT in psoas, and 0.02 (- 0.35, 0.38) for muscle in psoas. Compared to a conventional application, Tissue Compass demonstrated high accuracy and non-inferiority while also facilitating easier analyses. Tissue Compass could become the tool of choice to diagnose tissue loss/gain syndromes in the future by requiring a small number of CT sections to detect tissue volumes and fat infiltration.

Impaired complex theory of mind and low emotional self-awareness in outpatients with borderline personality disorder compared to healthy controls: A cross-sectional study.

Studies on the theory of mind (TOM) and alexithymia in borderline personality disorder (BPD) have yielded inconsistent results. Also, the relationship between TOM abilities and alexithymia facets as two domains of social cognition has not been studied in BPD. This study aimed to fill this gap. Participants were 50 outpatients with BPD and 50 age and gender-matched healthy controls. Assessments performed using Reading the Mind in Eyes Task (RMET), Toronto Alexithymia Scale (TAS-20), Faux Pas Task (FPT), and Digit Span subtest of Wechsler Adult Intelligence Scale. Results showed that BPD patients scored lower on overall FPT (p < .001) and its cognitive (p < .001) and affective TOM (p < .001) subtests but were comparable with healthy controls in emotion recognition ability assessed by RMET (p = .241). The BPD group also scored significantly lower in overall alexithymia (p < .001) and subscales of difficulty identifying feelings (DIF; p < .001) and difficulty describing emotions (DDF; p = .001). However, they performed similarly to the healthy control group in externally oriented thinking (EOT; p = .164). Correlation analysis revealed a significant negative correlation between EOT and RMET in the BPD group (r = -0.33, p < .05). No association, however, was found between FPT and RMET. This study suggests that BPD patients are impaired in the complex TOM abilities and have lower self-awareness of emotions, but their recognition of others' emotions is intact. Also, the results demonstrate that a heightened level of EOT is associated with difficulties in facial emotion recognition in BPD patients.

A meta-analysis and meta-regression of the effects of vitamin E supplementation on serum enrichment, udder health, milk yield, and reproductive performance of transition cows.

Vitamin E is an essential nutrient for cows, but the effect of vitamin E supplementation is often controversially discussed in the published literature. The main goal of this meta-analysis was to evaluate the effects of vitamin E supplementation on its serum and colostrum enrichment, milk yield (MY), and somatic cell counts (SCC), as well as on various reproductive variables of transition cows, by considering a large set of variables that might influence the responses to vitamin E supplementation. After a broad search in journals and databases with keywords related to transition cows supplemented with vitamin E and appropriate filtering of the results, 36 papers including 53 trials were selected, and their data were extracted into a database. A meta-analysis was conducted on the extracted data. The analysis showed enrichment of serum vitamin E both at parturition (effect size: 2.423) and postpartum (effect size: 0.473), but no effects of vitamin E supplementation on IgG concentration in colostrum (effect size: -0.05) were found. There was a tendency for supplemented cows to produce more milk (effect size: 1.29) during the first month of lactation. Because of large heterogeneity, a meta-regression was performed but none of the presumed influencing factors was identified as a potential variable affecting MY. Milk SCC, as an indicator of udder health, was unaffected by vitamin E supplementation. Vitamin E supplementation tended to decrease the calving to first estrus period (CFP), whereby supplementing Se and taking parity into account in the analysis significantly lowered the CFP. Cows receiving additional vitamin E had, on average, 6.1% fewer cases of retained placenta, whereby Se supplementation and breed were key factors improving the effect of vitamin E to reduce retained placenta. In this regard, breeds other than Holstein responded better and these cows showed a lower incidence of retained placenta. The supplemented cows showed fewer days open (effect size: -0.31), and this improvement was affected linearly by increasing the dosage administered. Also, cows showed fewer services per conception with increasing dosage of vitamin E. In conclusion, this analysis showed that supplementing vitamin E did not affect SCC or colostrum quality but improved reproductive performance of transition cows, an effect consistent with increased levels of serum vitamin E and, for some variables, being modulated by Se supplementation.

Adaptive Particle Filtering for Fault Detection in Partially-Observed Boolean Dynamical Systems.

We propose a novel methodology for fault detection and diagnosis in partially-observed Boolean dynamical systems (POBDS). These are stochastic, highly nonlinear, and derivativeless systems, rendering difficult the application of classical fault detection and diagnosis methods. The methodology comprises two main approaches. The first addresses the case when the normal mode of operation is known but not the fault modes. It applies an innovations filter (IF) to detect deviations from the nominal normal mode of operation. The second approach is applicable when the set of possible fault models is finite and known, in which case we employ a multiple model adaptive estimation (MMAE) approach based on a likelihood-ratio (LR) statistic. Unknown system parameters are estimated by an adaptive expectation-maximization (EM) algorithm. Particle filtering techniques are used to reduce the computational complexity in the case of systems with large state-spaces. The efficacy of the proposed methodology is demonstrated by numerical experiments with a large gene regulatory network (GRN) with stuck-at faults observed through a single noisy time series of RNA-seq gene expression measurements.

Bone Marrow Adipose Tissue Quantification by Imaging.

PURPOSE OF REVIEW: The significance and roles of marrow adipose tissue (MAT) are increasingly known, and it is no more considered a passive fat storage but a tissue with significant paracrine and endocrine activities that can cause lipotoxicity and inflammation. RECENT FINDINGS: Changes in the MAT volume and fatty acid composition appear to drive bone and hematopoietic marrow deterioration, and studying it may open new horizons to predict bone fragility and anemia development. MAT has the potential to negatively impact bone volume and strength through several mechanisms that are partially described by inflammaging and lipotoxicity terminology. Evidence indicates paramount importance of MAT in age-associated decline of bone and red marrow structure and function. Currently, MAT measurement is being tested and validated by several techniques. However, purpose-specific adaptation of existing imaging technologies and, more importantly, development of new modalities to quantitatively measure MAT are yet to be done.